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(54) Method and apparatus for signaling path restoration information in a mesh network 



(57) A method and apparatus are disclosed for mon- 
itoring and signaling a path restoration using pre-com- 
puted restoration paths following a detected fault on a 
primary service path in a communications network. A 
fault occurring inside the restorable portion of a network 
in heterogeneous or multiple network environments can 
be distinguished from faults occurring outside the restor- 
able network in accordance with the ANSI Tandem Con- 
nection Maintenance standard, T1 .105.05-1994. Path 
restoration is activated only when a fault causing path 
failure occurs inside the restorable portion of the net- 
work. Each conforming node in the restorable portion of 
the network has the necessary monitoring, signaling 
and cross-connect functionality and databases to par- 
ticipate actively in real time restoration. Additional non- 



conforming network elements can be positioned be- 
tween the restoration nodes without preventing path 
restoration. With the signaling architecture of the 
present invention, when an end-node detects a path fail- 
ure caused by an in-network fault, it formulates a sign- 
aling message for restoring the failed path. The resto- 
ration signaling message is thereafter relayed from one 
node to another in the overhead or payload of signaling 
paths that occupy the same bandwidth that is subse- 
quently used by the restoration path. Once a signaling 
message is transmitted to an adjacent node in the over- 
head or payload of a particular signaling path, the node 
that transmitted the message makes a cross-connect 
that replaces the signaling path with a segment of the 
restoration path whose set-up was requested in the 
transmitted signaling message. 
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Description 

Field of the Invention 

[0001] The present invention relates generally to 
techniques for restoring communication in a network af- 
ter a failure in a link or node of the network, and more 
particularly to techniques for signaling a path restoration 
using pre-computed restoration paths. 

Background of the Invention 

[0002] Mesh networks consist of nodes interconnect- 
ed by links. Mesh networks have long been used for a 
variety of communications applications, and the tech- 
nology for providing them has evolved overtime. Today, 
most large-scale mesh networks used for communica- 
tions applications are digital. In other words, the infor- 
mation being transported is encoded as a bit stream that 
the network nodes can access. Networks that use Syn- 
chronous Optical Network (SONET)/Synchronous Dig- 
ital Hierarchy (SDH) technology are examples of digital 
networks. A SONET line operating at a given transmis- 
sion (bit) rate may transport numerous multiplexed low- 
er-speed SONET paths. Mesh networks can also be op- 
tical. In an optical network, each optical line carries com- 
munications on numerous wavelengths. Recent ad- 
vances in optical technology are allowing the deploy- 
ment of large-scale optical mesh networks. 
[0003] Within a mesh network, end-to-end paths carry 
customer information from one customer location to an- 
other through a series of links and nodes. A node gen- 
erally provides a cross-connect function, routing a path 
from one line to another based on a map that is stored 
within the node's database. A node may also multiplex 
a number of paths together into a single higher rate sig- 
nal so that the paths can be transported efficiently 
through the network on a single link. At the next adjacent 
network node, the higher rate signal can be demulti- 
plexed, and the constituent paths cross-connected in- 
dependently, thus ensuring that each individual path is 
routed appropriately. 

[0004] In a SONET mesh network, for example, SON- 
ET Digital Cross-Connect Systems (DCSs) perform the 
functions of the network nodes. SONET lines, carried 
on fiber extending between two adjacent DCSs, provide 
the network links. SONET lines also connect a custom- 
er's SONET equipment to the network. Hence, a SON- 
ET path that originates and terminates in customer 
equipment is transported across the SONET mesh net- 
work via a series of SONET lines that interconnect SON- 
ET DCSs, as illustrated in FIG. 1 . FIG. 1 illustrates a 
path 1 1 0 in an exemplary SONET network 1 00 between 
two customer equipment (CE) devices 120, 130. As 
shown in FIG. 1 , SONET Path 1 originates and is for- 
matted in customer equipment E, enters the network at 
DCS A and is cross-connected (i.e., routed) at DCSs A, 
B and C. The path exits the network at node C and is 



terminated in customer equipment F In transiting the 
network, SONET Path 1 is transported via four distinct 
SONET lines (i.e., between nodes E and A, A and B, B 
and C, and C and F). When the path is bidirectional, both 

s directions of transmission would normally be routed via 
the same set of lines and nodes. 
[0005] In a SONET network, equipment originating 
paths and lines add overhead bits to the customer's pay- 
load (i.e., the information an end customer is sending or 

10 receiving). The overhead has a variety of uses, including 
for example, performance monitoring. In formatting Path 
1, customer equipment E adds SONET path overhead 
to the path payload as prescribed by SONET standards. 
When the path is subsequently terminated at customer 

15 equipment F, the path overhead is removed and proc- 
essed. SONET DCSs located at intermediate points 
along the path would not normally read or write path 
overhead. Instead, they pass the path payload and over- 
head through to the next node transparently. 

20 [0006] Nodes that originate and terminate SONET 
lines can multiplex a number of lower rate SON ET paths 
(including both payload and overhead) together onto a 
single higher speed SONET line so that the paths can 
be transported efficiently from one node to the next on 

25 a single fiber. SON ET line overhead is added to the mul- 
tiplexed signal by the node that originates the line. When 
the line is subsequently terminated at the downstream 
adjacent Line Terminating node, the line overhead is re- 
moved and processed, the signal is demultiplexed, and 

30 the constituent SONET paths are cross-connected in- 
dependently. As a result of the cross-connection, the 
constituent paths from a single incoming line may be 
routed and then multiplexed onto different outgoing 
lines. 

35 [0007] A number of important issues in the design of 
large-scale mesh networks relate to traffic restoration in 
the event of a link or node failure. A simple approach to 
restoration in a mesh network is to provide complete 
path redundancy, such that the network includes a ded- 

40 icated back-up or secondary path for each primary path 
of the network. FIGS. 2a and 2b illustrate a link failure 
and a node failure, respectively, in a portion of a bidirec- 
tional path 210. When there is a failure along the primary 
path 210, as illustrated in FIGS. 2a and 2b, customer 

4 5 traffic may then be transported on the secondary con- 
nection (not shown). Complete path redundancy Is the 
basis for the SONET 1 + 1 Path Switching, illustrated in 
FIG. 3. With SONET path switching, the customer's traf- 
fic is bridged onto both the primary and secondary paths 

so 31 o-1 , 31 0-2 at the node 320 where the customer traffic 
enters the network 300, creating a duplicate signal. The 
primary and secondary paths 310-1, 310-2 are kept 
node and link disjoint and are diversely routed through 
the network 300, but are brought back together at the 

55 node 330 where the customer's traffic leaves the net- 
work 300. A selector function 340 located in the egress 
node 330 monitors input from both the primary and sec- 
ondary paths 310-1 , 310-2 and selects the better of the 
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duplicated signals to forward to the customer's location 
350. When there is a failure in a link or node that affects 
one path, the selector 340 automatically selects the sig- 
nal being forwarded to the customer from the other bet- 
ter path. For a detailed discussion of SONET path 
switching applications see; for example, "SONET Dual- 
Fed Unidirectional Path Switch Ring (UPSR) Equipment 
Generic Criteria", Telcordia GR-1400-CORE Issue 2, 
January 1999, incorporated by reference herein. 
[0008] Unfortunately, providing dedicated redundant 
paths uses a large amount of restoration bandwidth, 
making 1+1 path selection costly and undesirable for 
many networks. More sophisticated algorithmic ap- 
proaches to path restoration allow multiple paths to 
share part or all of the same restoration bandwidth 
whenever possible. When a primary service path fails, 
the nodes in the network act under software control to 
make cross-connects that set up a secondary path in 
the restoration bandwidth and route the customer's traf- 
fic onto it. If a second primary path that shares restora- 
tion bandwidth with the first path subsequently fails be- 
fore the first path is repaired, the second failed path can- 
not be restored using that bandwidth. 
[0009] Algorithmic approaches resulting in shared 
restoration bandwidth fall into two broad categories, 
namely, Distributed, Discovery-based Techniques and 
Techniques Using Pre-Computed Paths. Distributed, 
Discovery-based Techniques identify and activate res- 
toration paths during a real-time search that is initiated 
by a network node after detecting the failure of a sub- 
tended link. Essentially, when a node detects a link fail- 
ure, it contacts other nodes to identify spare capacity on 
other non-failed links that are potential candidates for 
alternate routing. The available spare capacity is allo- 
cated link-by-link on a first-come-first-served basis. Be- 
cause it is the nodes at the ends of a failed link that in- 
itiate the search for restoration capacity, distributed dis- 
covery-based techniques are fundamentally intended 
for restoration from single link failures in networks where 
failed links can be identified by the nodes that terminate 
them. In SONET networks, line-terminating nodes are 
capable of isolating line failures; hence distributed, dis- 
covery-based techniques can be used for recovering 
from some failures. However, distributed, discovery- 
based techniques do not perform well when there is a 
node failure, and generally cannot be used by multiple 
nodes simultaneously. For a detailed discussion of such 
distributed discovery- based computation approaches, 
see, for example, W. D. Grover, "The Self-Healing Net- 
work: A Fast Distributed Restoration Technique for Net- 
works Using Digital Cross Connect Machines," IEEE 
Globecom 1 987, and U.S. Patent No. 4,956,835, issued 
to W. D. Grover on September 11 , 1990, each incorpo- 
rated by reference herein. 

[0010] Techniques Using Pre-Computed Paths iden- 
tify (or pre-compute) restoration paths in anticipation of 
network failures. The pre-computed restoration paths, 
however, are activated only when triggered by an actual 



failure event. The key advantage of using pre-computed 
restoration paths over discovery-based techniques is 
that, because there is no pressure to make a real-time 
selection of a restoration path, the restoration algorithm 

5 can take more time to optimize the use of the restoration 
bandwidth. Hence, for any given network failure, more 
paths are likely to be restored and bandwidth used more 
efficiently. In addition, in the event of a failure, network 
restorations can be completed faster since there is no 

10 need to search for restoration paths. 

[0011] In techniques using pre-computation, the pre- 
computation may be either centralized or distributed. In 
a centralized computation , a central controller/database 
for the network stores information on the entire network 

*5 topology including the amount of spare capacities of all 
links in the network. With this information as input, the 
central controller/database runs an algorithm with the 
objective of computing restoration paths for each prima- 
ry service path in the network. As output, the controller 

20 creates a routing table that specifies which cross-con- 
nects (or equivalent information) are to be made at net- 
work nodes to restore customer service when there is a 
failure in the network. The routing table may be stored 
within the controller/database, or it may be partitioned 

25 into multiple routing tables each including only the 
cross-connects to be made at a particular node. In the 
latter case, the partitioned tables are then downloaded 
to their respective network nodes where they are stored 
until needed to effect a restoration. 

30 [0012] Different strategies are required for activating/ 
controlling restoration, depending on whether the rout- 
ing table is stored in the controller or in the network 
nodes. In the former case, the network node or nodes 
that detect the failure notify the controller. On receiving 

35 this information , the controller accesses its routing table 
and, based on the information it receives from the de- 
tecting nodes, issues cross-connect commands to the 
network nodes that must take action to restore service. 
This method is called centralized computation with cen- 

40 tralized activation/control of restoration. In the latter 
case, when routing tables are stored locally in each net- 
work node, the nodes that detect a failure notify the 
nodes that must take action to restore service directly, 
or the notification is relayed from node to node in the 

45 network. On receiving a failure notification, each node 
accesses its local routing table and, based on informa- 
tion received in the notification, executes the appropri- 
ate cross-connects needed locally to restore service. 
This method is called centralized computation with dis- 

so tributed activation/control of restoration. For a more de- 
tailed discussion of centralized pre-computation tech- 
niques, see, for example, J. Anderson, B.T Doshi, S. 
Dravida and P. Harshavardhana, "Fast Restoration of 
ATM Networks," JSAC 1 991 , incorporated by reference 

55 herein. 

[0013] In a distributed pre-computation, the computa- 
tion of the restoration routes is distributed among the 
nodes in the network, each of which has information 
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concerning capacities of the links it terminates. During 
the computation, each node creates a routing table with 
a local view of the restoration paths to be used in the 
event of path failures. The routing table is stored within 
the respective network node. Subsequently, when there 
is a failure in the network, the restoration actions of the 
nodes are similar to those described above for distrib- 
uted control/activation of restoration. However, because 
the computation of restoration paths is distributed 
among the nodes of the network, this method is referred 
to as distributed computation with distributed control/ac- 
tivation of restoration. 

[0014] United States Patent Application Serial 
Number 08/960,462, filed October 29, 1997, entitled 
"Distributed Pre-computation of Signal Paths In An Op- 
tical Network," incorporated by reference herein, dis- 
closes improved network restoration techniques, re- 
ferred to hereinafter as the "Pre-computed Restoration 
Techniques." The disclosed Pre-computed Restoration 
Techniques utilize distributed pre-computation to pro- 
vide path restoration in large-scale optical mesh net- 
works after a link, span or node failure while, at the same 
time, allowing multiple paths to share restoration band- 
width. Each restoration path is pre-computed to be 
physically disjoint and diversely routed from the associ- 
ated primary path, except for the end nodes providing 
access and egress to the network. The Pre-computed 
Restoration Techniques allow a single restoration path 
to protect a given primary service path. Hence, no mat- 
ter which node or link fault causes a path failure, the 
path is always restored in the same way. Once a failure 
is detected in one or more primary service paths, the 
pre-computed restoration paths can be activated in a re- 
al-time manner. 

[0015] The disclosed Pre-Computed Restoration 
Techniques provide methods for distributed pre-compu- 
tation of end-to-end restoration paths and allow distrib- 
uted real-time restoration in optical mesh networks. 
They can also be applied without modification to pre- 
computing end-to-end restoration paths for SONET/ 
SDH mesh networks. However, they do not address the 
signaling that the network nodes must use after a failure 
to activate and control a distributed real-time restoration 
in either an optical or a SONET/SDH network when the 
Pre-Computed Restoration Techniques have been used 
to compute the restoration paths. 
[0016] Signaling methods can be designed to use a 
signaling network having links and nodes that are phys- 
ically separate from the links and nodes of the mesh net- 
work, except where a signaling network link interfaces 
physically to a mesh network node. The physical sepa- 
ration limits the impact of mesh network failures on the 
ability to signal when a mesh network restoration is re- 
quired. Such physically separate networks are often 
used for restoration signaling when both pre-computa- 
tion and activation/control are centralized. Such net- 
works are often fully duplexed to provide high reliability. 
[001 7] A separate, reliable signaling network could al- 



so be used for node-to-node communication in a distrib- 
uted restoration. However, the operational complexity of 
constructing, provisioning and maintaining a separate 
signaling network makes using a separate network un- 

5 desirable for many restoration applications. For such 
applications, it is preferable to transport signaling 
through the mesh network itself, provided it can be done 
reliably and cost-effectively. Reliable transport means 
that the specific links and nodes of the mesh that are 

10 used for restoration signaling must be available when 
needed, in other words, they cannot be affected by the 
mesh network failure that necessitated restoration sig- 
naling in the first place. Within the mesh, reliability for 
signaling paths can be provided with complete path re- 

15 dundancy. However, as noted earlier, providing dedicat- 
ed redundant paths, whetherfor reliability or restoration, 
uses a large amount of bandwidth, which tends to be 
costly. Hence, a need exists for a method that allows 
sharing or reuse of signaling bandwidth, while at the 

20 same time provides reliability for signaling. 

[0016] An additional concern in using the mesh net- 
work itself for signaling is that, within existing networks, 
for example, in SONET networks that are already widely 
deployed, there may heterogeneous network elements, 

25 such as network elements with diverse monitoring, sig- 
naling and cross-connect functionality and databases. 
For example, the network may include older generation 
network elements of a given manufacturer, or network 
elements provided by a number of manufacturers, that 

30 each provide varying restoration capabilities, if any. A 
need therefore exists for a signaling method and appa- 
ratus that permits the restoration of a failed primary serv- 
ice path, even in the presence of such non-conforming 
network elements. 

35 

Summary of the Disclosure 

[0019] Generally, a method and apparatus are dis- 
closed for monitoring for primary path failures and sig- 
40 naling path restorations using pre-computed restoration 
paths following the failure of a link or node within in any 
mesh network, such as a SONET mesh network, in 
which the restoration nodes can (i) originate and termi- 
nate paths; and (ii) read restoration-related information 
45 from and write restoration- related information into path 
overhead or payload as described In this document. 
Pre-computed restoration paths compatible with the dis- 
closed methodology can be obtained, for example, in 
accordance with the Pre-computed Restoration Tech- 
no niques described earlier. A network that implements the 
present invention will be referred to as a "restorable net- 
work." Each of the conforming nodes in a restorable net- 
work is referred to as a restoration node and has the 
necessary monitoring, signaling and cross-connect 
55 functionality and databases to participate actively in a 
real time restoration in accordance with the present in- 
vention. In addition, non -conforming network elements, 
such as those without the necessary functionality and 
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databases, can be positioned in between the restoration 
nodes and do not prevent restoration in accordance with 
the present invention. 

[0020] Within a restorable SONET network, a primary 
path is assumed to be coincident with a SONET path. 
However, end-to-end SONET paths can extend beyond 
the boundaries of a restorable network, for example, to 
other SONET networks operated by different adminis- 
trations (where restoration techniques may or may not 
be implemented), or to customer equipment that is not 
considered part of the network. A SONET path can fail 
as a result of a node or link failure occurring either inside 
or outside the boundaries of the restorable network. 
However, the present invention triggers path restoration 
signaling only when a fault causing a path failure occurs 
within the identified boundaries of the restorable net- 
work. Such failures are referred to as "in-network" faults. 
Hence, according to one aspect of the disclosed inven- 
tion, a mechanism is provided for monitoring each path 
traversing the restorable network for a path failure and, 
when such a failure is identified, determining whether or 
not the failure is caused by a fault occurring inside or 
outside of the restorable network. In the disclosed in- 
vention, the nodes where a SONET path enters and ex- 
its the restorable network, referred to herein as "end 
nodes," monitor for path failures and subsequently trig- 
ger restoration signaling when required. End nodes for 
each service path are identified when the service path 
is initially provisioned. 

[0021] In an illustrative implementation, the determi- 
nation of whether a fault occurs inside or outside of a 
network is performed in accordance with the well-known 
ANSI Tandem Connection standard. Thus, the present 
invention is able to operate in an environment where the 
path terminations are located outside the restorable net- 
work, e.g., in a multi-network environment or where cus- 
tomer path terminating equipment is not part of the 
restorable network, and to trigger restoration signaling 
only when the fault causing a path failure is located with- 
in the restorable network. 

[0022] Under the Pre-computed Restoration Tech- 
niques, multiple primary service paths potentially share 
all or some of the same restoration bandwidth. Because 
two paths cannot be provisioned in the same bandwidth 
at the same time, restoration paths cannot be provi- 
sioned before a failure occurs and must be set up after 
an "in-network" failure has been detected. Thus, a rapid, 
robust and reliable signaling method is required to trans- 
mit information about the path failure from the end nodes 
that detect the failure to the restoration nodes that must 
perform cross-connects to restore service. . The present 
invention provides for node-to-node signaling to enable 
distributed restoration of the network. The node-to-node 
signaling of the present invention aims to enable sub- 
second restoration in large carrier-grade networks (giv- 
en reasonable assumptions about the cross-connect 
rate of the restoration nodes and the numbers of cross- 
connect commands that must be processed by each 



node during a typical restoration), use bandwidth effi- 
ciently and operate reliably, even in the presence of non- 
conforming SONET network elements located between 
restoration nodes. 

5 [0023] The present invention uses a multiple replace- 
able paths architecture for node-to-node signaling. With 
the disclosed multiple replaceable paths architecture, 
two adjacent restoration nodes create SONET paths for 
restoration signaling in the restoration (or "spare") band- 

10 width that lies between them. The restoration nodes 
originate and terminate these SONET paths. These 
paths extend between the restoration nodes and pass 
transparently through any intervening non-conforming 
SONET network elements because non-conforming 

is network elements do not terminate the paths. Subse- 
quently, when an end node of a primary path detects an 
"in-network" path failure, the end node formulates a sig- 
naling message that identifies the failed path uniquely 
and requests set-up of the restoration path. The resto- 

20 ration signaling message is thereafter relayed from one 
restoration node to another, for example, using the over- 
head or payload of the signaling paths that occupy the 
exact same bandwidth that will subsequently be used 
by the restoration path. 

25 [0024] When the pre-computed restoration path pass- 
es through at least three (3) restoration nodes, including 
the end nodes, several signaling paths will be used in 
tandem to signal a restoration. In this case, the interme- 
diate restoration nodes make signaling routing deci- 
de sions based on information from the pre-computation 
which they have stored in their routing tables. Once a 
signaling message is transmitted to an adjacent node 
using the overhead or payload of a particular signaling 
path, the node that transmitted the message makes a 

35 cross-connect that replaces that signaling path with a 
segment of the restoration path whose set-up was re- 
quested in the transmitted signaling message. When the 
signaling message has passed through all intermediate 
nodes on the restoration path and reaches the far-end 

40 end node, and the far-end node verifies end-to-end con- 
nectivity and makes its final cross-connect, the failed 
path is restored. 

[0025] The invention provides the following benefits: 
Because a signaling path follows the same route and 

45 occupies the same bandwidth as (a segment of) a pre- 
computed restoration path, if a pre-computed restora- 
tion path is available, the paths for signaling its set-up 
are also available. Hence, the method is reliable. In ad- 
dition, because signaling messages are carried in the 

50 restoration bandwidth, no additional bandwidth needs 
to be dedicated for signaling. Hence, the method uses 
bandwidth efficiently. Also, because the signaling paths 
extend between the restoration nodes and pass trans- 
parently through any intervening non-conforming SON- 

55 ET network elements, signaling messages transported 
in those paths also pass transparently through interven- 
ing non-conforming network elements. Hence, the sig- 
naling method will operate even in the presence of such 
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non-conforming network elements. 
[0026] A more complete understanding of the present 
invention, as well as further features and advantages of 
the present invention, will be obtained by reference to 
the following detailed description and drawings. 

Brief Description of the Drawings 

[0027] 

FIG. 1 illustrates a SONET path between two client 
equipment (CE) devices in an exemplary network 
where the present invention can operate; 
FIG. 2a shows a link failure in a portion of a bidirec- 
tional path; 

FIG. 2b shows a node failure in a portion of a bidi- 
rectional path; 

FIG. 3 illustrates 1+1 Path Protection in a SONET 
network; 

FIG. 4 illustrates an exemplary SONET network in 
which the present invention can operate; 
FIG. 5 illustrates one of the nodes from the network 
of FIG. 4; 

FIG. 6 illustrates the issuance of maintenance sig- 
nals by network elements in response to a failure 
that affect both directions of a transmission along a 
given path; and 

FIG. 7 illustrates the signaling of pre-computed path 
information along signaling paths using the path 
overhead or payload in accordance with the present 
invention. 

Detailed Description 

[0028] FIG. 4 illustrates an exemplary network 400, 
such as a SONET network, in which the present inven- 
tion can operate. Generally, the network 400 includes at 
least three conforming nodes, such as nodes A, B and 
D, each referred to as restoration nodes, and has well- 
defined end-to-end paths, such as Path 1 and Path 2. It 
is noted that each of the nodes shown in FIG. 4 are con- 
forming nodes. In addition, non-conforming network el- 
ements (not shown), such as those without the neces- 
sary functionality and databases, can be positioned in 
between the restoration nodes and do not prevent res- 
toration in accordance with the present invention. The 
present invention may be employed in any mesh net- 
work in which the restoration nodes can (i) originate and 
terminate paths, and (ii) read restoration-related infor- 
mation from and write restoration-related information in- 
to path overhead or payload as described in this docu- 
ment 

[0029] Generally, when a node does not terminate a 
path in a SONET network, the transmitted information 
(payload and overhead) is passed transparently through 
the node. The restoration nodes of the present invention 
can access the path overhead as it passes through a 
node. In addition to the illustrative SONET networks, this 



method would be applicable, for example, in networks 
with Synchronous Digital Hierarchy (SDH) technology, 
or potentially in optical networks with paths having as- 
sociated digital information, such as a digital wave- 

5 wrapper, since wave-wrapper technology encloses 
each optical wavelength/path in a digital "wrapper" that 
can be accessed by intermediate nodes along the path. 
[0030] The present invention provides monitoring and 
signaling capabilities for the implementation of real-time 

10 distributed path restoration in the exemplary SONET 
network 400 using pre-computed restoration paths. The 
pre-computed restoration paths can be obtained, for ex- 
ample, in accordance with the Pre-computed Restora- 
tion Techniques referenced above. According to one 

is feature of the present invention, pre-computed restora- 
tion paths are activated in response to a detected fault 
on a primary service path and customer service is there- 
by restored when there is an in-network fault. In the mul- 
tiple network environment of the present invention, it is 

20 important to determine if a fault occurs within the restor- 
able portion of the network 400, referred to as an "in- 
network" fault. Otherwise, it is possible to activate cross- 
connects that will not result in restoration of the failed 
service because the fault causing the problem is outside 

25 the restorable network. Later, if another path that shares 
restoration bandwidth with the first failed path also fails 
due to an "in-network" fault, it will not be possible to re- 
store it because its restoration bandwidth has already 
been claimed for the unsuccessful restoration of the first 

30 path. 

[0031] Thus, according to another feature of the 
present invention, a mechanism is provided for deter- 
mining whether or not a fault occurs inside or outside of 
a restorable network, and for activating path restoration 
35 signaling only when the fault that caused path failure is 
inside the restorable network. In one implementation, 
discussed further below, the determination of whether a 
fault occurs inside or outside of a network is performed 
in accordance with the ANSI Tandem Connection stand- 
ee ard, described in "Synchronous Optical Network (SON- 
ET) - Tandem Connection Maintenance," ANSI 
T1 .105.05, incorporated by reference herein. 
[0032] Generally, in a SONET network, a network 
fault is identified when a network node detects Loss Of 
45 Signal (LOS) on an incoming SONET line. LOS may be 
due to a failure of the line itself or of the node at the other 
end of the line. The detecting node transmits a standard 
SONET maintenance signal, Alarm Indication Signal- 
Path (AIS-P), away from the failure in all affected paths 
50 carried on that line. When the failure is bi-directional, 
the AIS-P signal propagates in both directions from the 
nodes adjacent to the failure to the terminations of the 
respective SONET paths. In a SONET network, the AIS- 
P is an all ones signal transmitted in the path payload, 
55 path overhead and the path pointers. To detect path fail- 
ure, it is generally sufficient to monitor the pointers 
alone. 

[0033] Under the ANSI Tandem Connection Mainte- 
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nance standard, a restorable in -network fault is differ- 
entiated from a general network fault by the end-nodes 
along a given path. When an end node detects a failure 
of a SONET path entering the restorable network from 
the outside, that end node re-establishes the SONET 
path pointers so that the Path Overhead can be ac- 
cessed. It then places a flag in the path overhead to in- 
dicate that a failure was detected on the path as it en- 
tered the network. With the path pointers thus re-estab- 
lished, out-of-network faults become transparent to all 
the subsequent nodes along a given path. Subsequent 
nodes see valid pointers and treat the SONET paths as 
if they were carrying valid user traffic, instead of the 
characteristic all ones signal of AIS-P. Hence, these 
subsequent nodes do not attempt to initiate path resto- 
ration. However, the presence of the flag in the path 
overhead triggers the exit end node (the far-end node 
where the path leaves the network) to re-insert AIS-P 
on the out-bound path, thus assuring that downstream 
SONET Path Terminating Equipment (PTE) lying out- 
side the restorable network knows about the original fail- 
ure. On the other hand, when an exit end node finds 
AIS-P on an out-bound path, as indicated by an invalid 
path pointer, then the fault causing the path failure oc- 
curred within the restorable network. The node acts on 
this information and subsequently triggers restoration 
signaling. 

[0034] Another aspect of the present invention ad- 
dresses the required node-to-node signaling to enable 
distributed restoration of the exemplary network 400. 
Generally, the node-to-node signaling aims to enable 
sub-second restoration in large carrier-grade networks, 
use bandwidth efficiently, operate reliably and also be 
compatible with non-conforming network elements. As 
discussed further below, within the signaling architec- 
ture of the present invention, one or both end nodes of 
a failed path within the restorable network formulate a 
signaling message requesting restoration. The restora- 
tion-signaling message is thereafter relayed from one 
restoration node to another in the overhead or payload 
of signaling paths that occupy the exact same band- 
width that is subsequently used by the restoration path. 
Restoration signaling messages are passed through 
non-conforming nodes transparently since those nodes 
do not terminate the signaling paths. As discussed fur- 
ther below, various embodiments of the present inven- 
tion provide for restoration path set-up, path removal, 
and handling of misconnecttons and priorities. 
[0035] FIG. 4 shows an exemplary SONET network 
400 in which the restoration techniques of the present 
invention may be implemented. As shown in FIG. 4, the 
SONET network 400 includes a boundary 41 0 that sep- 
arates the SONET network 400 into a restorable portion 
and a non-restorable portion. The SONET network 400 
includes a number of nodes 420-428 in the restorable 
portion of the network 400 and a number of nodes 
430-433 in the non- restorable portion of the network 
400. Each of the nodes 420-428 in the restorable portion 



of the network 400, referred to hereinafter as restoration 
nodes 420-428, may be embodied as a SONET Digital 
Cross-Connect System (DCS), discussed further below 
in conjunction with FIG. 5, as modified herein to provide 
5 the features and fu notions of the present invention. Each 
of the restoration nodes 420-428 have the necessary 
monitoring, signaling and cross-connect functionality 
and databases to participate actively in real time resto- 
ration in accordance with the present invention. For a 
10 more detailed discussion of SONET DCSs and the 
structure of SONET signals, including line and path 
overhead, as well as generic monitoring of SONET sig- 
nals, see, for example, Generic Criteria for SONET Dig- 
ital Cross-Connect Systems (DCS), Telcordia GR- 

15 2996-CORE, Issue 1 , January, 1 996; and Synchronous 
Optical Network (SONET) Transport Systems: Common 
Generic Criteria, Telcordia GR-253-CORE, Issue 2, Re- 
vision 2, 1999, each incorporated by reference herein. 
[0036] It is noted that there may be additional non- 

20 conforming network elements (not shown) located be- 
tween the restoration nodes 420-428. The non-conform- 
ing network elements may be, for example, older gen- 
eration network elements of a given manufacturer, or 
network elements provided by a number of manufactur- 
es ers. The non-conforming network elements do not pro- 
vide the necessary monitoring, signaling and cross-con- 
nect functionality and databases to participate actively 
in real time restoration in accordance with the present 
invention. However, according to a feature of the 

30 present invention, discussed further below, the restora- 
tion techniques of the present invention work even in the 
presence of such non-conforming network elements. 
[0037] It is again noted that a given path can traverse 
a plurality of networks in the multiple network environ- 

35 ment of the present invention. Thus, a fault resulting in 
failure of a given path could occur in any of the networks. 
The restoration techniques of the present invention, 
however, enable service to be restored only if the fault 
causing path failure occurs within the boundary 410 of 

40 the "restorable network" and prevent unnecessary sig- 
naling or cross-connecting when the fault is outside the 
network. Thus, as previously indicated, the present in- 
vention utilizes a mechanism for detecting path failures 
and activating restoration that distinguishes a failure 

^5 that occurs within the restorable network from a failure 
that occurs outside the restorable network. 
[0038] FIG. 4 illustrates two disjoint paths 440, 450. 
Path 440 enters the boundary of the restorable network 
at node A and passes through nodes B and C before 

50 exiting the network at node D. Nodes A and D are re- 
ferred to as end nodes, since they mark the ends of the 
path within the restorable network. Nodes B and C are 
called intermediate nodes. Similarly, Path 450 enters 
the restorable network at end node E, passes through 

55 intermediate node F and exits at end node G. Within the 
restorable network, Path 440 and Path 450 are disjoint 
and diversely routed. Hence, they will not fail simultane- 
ously unless there are multiple faults in the network. Un- 
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der these conditions, the pre-computation algorithm of 
the present invention allows Path 440 and Path 450 to 
share all or part of the same restoration bandwidth, with 
a view that if both primary paths fail simultaneously, only 
one can be guaranteed to be restored. In this example, 
the pre-computed restoration path for Path 440 extends 
through nodes A, H, I and D. The pre-computed resto- 
ration path for Path 450 extends through E, H, I and G. 
The restoration bandwidth shared by these paths is be- 
tween nodes H and I. 

[0039] The sharing of restoration bandwidth allows 
network capacity to be used more efficiently and cost- 
effectively than with rings or 1+1 path protection 
schemes. However, when restoration bandwidth is 
shared, the restoration path cannot be provisioned be- 
fore a failure occurs. Thus, the restoration path must be 
set up (i.e., the appropriate cross -connects must be 
made at nodes along the restoration path) after the fail- 
ure has occurred. Thus, a rapid, robust and reliable sig- 
naling architecture is required to convey information 
about the fault from the place where the fault occurred 
and was detected to the nodes that must take action to 
restore service. 

[0040] It is noted that an end-to-end path between 
Path Terminating Equipments (PTEs) may cross one or 
more networks, and one or more of these networks may 
be restorable. As used herein, a restorable path extends 
from end node to end node within a restorable network. 
Hence, a part/segment of the overall end-to-end path 
traversing one of the restorable networks may be restor- 
able. 

[0041] The nodes 420-428 and 430-433 of the illus- 
trative network 400 are interconnected by optical fiber 
connections. (This connection may be direct from res- 
toration node to restoration node, or via intervening, 
non-conforming network elements that do not provide 
the restoration capabilities discussed in this invention.) 
It should be noted that the network 400 of FIG. 4 is sim- 
plified for purposes of illustration. The invention is well 
suited for use in large-scale regional, national and inter- 
national networks which may include many sub-net- 
works, each having hundreds of nodes. In a SONET net- 
work, for example, one or more of the sub-networks may 
be associated with each local exchange earner (LEC) 
and inter-exchange carrier (IXC) of the network. 
[0042] FIG. 5 shows one of the restoration nodes, 
such as node 420, of network 400 in greater detail. The 
node 420 includes a cross-conned fabric 58-i, and is 
connected to other nodes in the network 400 by means 
of bidirectional links 64 and 66and interfaces 70-1 , 70-2 
70-3, 72-1 , 72-2, and 72-3. The node 420 supplies SON- 
ET signals to the other nodes in the network 400 via the 
bidirectional links 64 and 66. The interfaces 70-1 , 70-2 , 
70-3, 72-1 , 72-2, and 72-3 provide optical/electrical con- 
version for signals on bidirectional links 64 and 66. The 
interfaces 70-1, 70-2, 70-3, 72-1, 72-2, and 72-3 also 
provide SONET line terminating functions, SONET path 
terminating functions for signaling paths, and read/write 



access to SONET path overhead or pay load for signal- 
ing, service and restoration paths, as described later. 
The node 420 also includes a control and memory func- 
tion 77 that may be provided on one central processor 
5 or multiple distributed processors in the node. A map of 
the current state of the fabric is included in the control 
and memory function 77. The control and memory func- 
tion 77 also contains routing tables that specify map or 
cross-connection changes needed to implement the 
pre-computed restoration paths. These same tables are 
also accessed to route signaling messages, as de- 
scribed later. The node 420 has been simplified for pur- 
poses of illustration, and as noted above may include a 
substantially larger number of input and output links, as 
required for a given application. 

REAL TIME RESTORATION 

[0043] The present invention preferably utilizes the 
Pre-computed Restoration Techniques referenced 
above to p re-compute a restoration path, from an end 
node, such as node 420, to an end node, such as node 
423, for each primary path, such as path 440, that 
traverses the restorable network. The Pre-computed 
Restoration Techniques are also described in B.T Doshi 
et al., "Optical Network Design and Restoration," Bell 
Labs Technical Journal (April-June 1999), incorporated 
by reference herein. With the Pre-computed Restoration 
Techniques, a single restoration path is pre-computed 
for each primary service path in the network. The resto- 
ration path passes through both end nodes of the asso- 
ciated primary service path , but is otherwise disjoint and 
diversely routed from the primary service path. Hence, 
no single failure, other than a failure of an end node, can 
cause both the primary path and the pre-computed res- 
toration path to fail simultaneously. In addition, no matter 
what the cause of a path failure (e.g., cable cut, node 
failure, equipment failure) when a given primary path 
fails : it is always restored using the same restoration 
path. The present invention provides the real-time ca- 
pability to determine which primary paths have failed so 
that they can be restored. 

[0044] Information describing pre-computed restora- 
tion paths for every primary service path in the network 
is stored in a database or databases until it is needed 
to restore service after an "in-network" failure. As noted 
earlier, this information can be stored either in a central 
controller/database for the network, or it can be parti- 
tioned and stored locally in each node. In the latter case, 
each node has a database containing a local view of the 
restoration paths. In either case, when there is a subse- 
quent node or link failure in the network, the paths af- 
fected by the failure must be identified, and the associ- 
ated restoration paths must be activated. The method 
for identifying the failed paths and activating the resto- 
ration paths depends on whether restoration path infor- 
mation is stored in a central controller/database or 
stored locally in each node. The present invention is ap- 
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plicable when data is stored locally in each network 
node. The process is referred to as distributed activa- 
tion/control of restoration. Hence., to implement the 
present invention, data from the pre-computation must 
be previously stored in the restoration nodes 420-428. 
The data stored in the restoration nodes is the same re- 
gardless of whether the pre-computation is done cen- 
trally or distributed among the nodes. 
[0045] One aspect of the disclosed invention is that 
the restoration nodes 420-428 will incorporate signaling 
functionality enabling node-to-node communication 
about failures. Because the end nodes, such as nodes 
420 and 423, are on both the primary and restoration 
paths, the end nodes have been selected to monitor for 
primary service path failures and to initiate node-to- 
node restoration signaling when necessary. Another as- 
pect of the disclosed invention is that restoration sign- 
aling will follow the route of the restoration path. This 
approach to restoration signaling is called restoration 
path signaling. 

[0046] The choice of signaling along the restoration 
path route is significant. Because the restoration path is 
disjoint and routed diversely from the primary service 
path, with the exception of the end nodes, no single fail- 
ure (except end-node failure) can affect both. Hence, if 
restoration signaling follows the route of the pre-com- 
puted restoration path from the end-nodes through the 
network, the signaling will get through to the nodes that 
must perform cross-connects to restore service, unless 
there are multiple simultaneous network failures. Con- 
versely, if bandwidth along the restoration route is not 
available, e.g., due to a second network failure, service 
could not be restored, even if another signaling method 
was used and restoration messages reached the appro- 
priate nodes via another route. 

[0047] Real-time restoration path signaling requires 
(i) capabilities at the end nodes to monitor and detect 
failures of the primary path and to initiate restoration sig- 
naling; (ii) capabilities in intermediate nodes to receive, 
process and forward signaling information on to the next 
node involved in the restoration; (iii) available paths 
along the restoration route to transport signaling; (iv) ca- 
pabilities at end nodes to verify that restoration connec- 
tions have been made properly, and to initiate backout 
signaling in the event of misconnections or other anom- 
alies; and (v) local storage (within each node) of infor- 
mation/data from the pre-computation needed to per- 
form the above functions. 

RESTORABLE NETWORK BOUNDARY 

Out of Network Failures Versus In Network Failures 

[0048] FIG. 6 illustrates how the SONET network el- 
ements 420-428 issue maintenance signals in response 
to failures that affect both directions of transmission 
along given paths 1 , 2. In this example, path 1 is carried 
on a SONET line 610 extending between nodes B and 



C. As illustrated in FIG 6, the line 610 fails (e.g., due to 
a cable cut). Hence, all paths carried on the line 61 0 fail 
as well. Nodes B and C detect a Loss of Signal (LOS) 
on the line 610 and transmit a standard maintenance 
5 signal, Alarm Indication Signal-Path (AIS-P), outward in 
all affected SONET paths/including Path 1 . The AIS-P 
signal propagates in both directions to the terminations 
of the SONET paths. The AIS-P is an all-ones signal 
that over-writes the path pointers, path overhead and 

10 path payload. Path AIS would normally be detected by 
monitoring for an all-ones pointer. 
[0049] The AIS-P traverses the length of the SONET 
path from the node adjacent to the failure, through the 
end node and beyond to SONET Path Terminating 

*5 Equipment. Thus, AIS-P could potentially be used as an 
indicator of primary path failure, and hence used to trig- 
ger restoration. However, there is a problem with using 
AIS-P alone for this purpose. A node that detects an in- 
coming AIS-P cannot distinguish between a SONET 

20 path failure that occurs in the restorable network, and 
one that occurs outside the restorable network, as illus- 
trated in FIG. 6, In the example shown in FIG. 6, path 1 
has an in-network failure and path 2 has an out-of-net- 
work failure. From the standpoint of end nodes D and 

25 g, however, these failures appear the same. The resto- 
ration techniques of the present invention, however, 
should be triggered only for in-network failures. As pre- 
viously indicated, the illustrative embodiment deter- 
mines whether a given failure is an in-network or out-of- 

30 network failure using the functionality specified in the 
ANSI Tandem Connection Maintenance standard. 

Out of Network Signal Degrade Versus In Network 
Signal Degrade 

35 

[0050] If in addition to hard failure conditions (also 
called Signal Fail conditions), Signal Degrade (which is 
characterized by excessive errors) is also to be used as 
a criteria for activating restoration, the Tandem Connec- 

40 tion Maintenance Standard also provides functionality 
for determining the number of bit errors per unit of time 
that occurred within the restorable network versus out- 
side the restorable network. Data on the number of in- 
network errors can be used as the basis for in-network 

45 signal degrade calculations, as would be apparent to a 
person of ordinary skill in the art. 

Restoration Node Capabilities for Identifying Failed 
Paths 

50 

[0051] The capabilities referenced above for identify- 
ing in- and out- of -network signal failures and degrades 
consist of a subset of the capabilities described in the 
Tandem Connection Maintenance Standard, ANSI 
55 T1 .105.05-1994, incorporated by reference herein. To 
provide these functions, the restoration nodes 420-428 
must have read and write access to the H1 , H2 and H3 
Path Pointers, the B3 parity byte, and the Z5 SONET 
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Tandem Connection Overhead byte. 
[0052] The restoration nodes 420-428 must also pro- 
vide new provisioning capabilities that allow the user to 
identify the two end nodes where a SONET path enters/ 
exits the restorable network. The provisioning of the 5 
path end nodes is necessary to clearly delineate the part 
of the SONET path that is contained within the restora- 
ble network and to activate the special path monitoring 
capabilities described above. These same end nodes 
must also initiate restoration signaling when they detect 10 
an in-network path failure. 

Modularity of Restoration 

[0053] The present invention can be implemented in is 
networks having various cross-connect rate signals, 
such as STS-1 , STS-3c, STS-1 2c and STS-48c rate sig- 
nals. Networks where most paths are STS-48cs or STS- 
. 1 2cs will generally restore faster than a network of equal 
size where the paths are STS-3cs or STS-1 s. This hap- 20 
pens because to restore an STS-48c path takes about 
as many cross-connects as to restore an STS-1 path, 
but restoring the STS-48c path restores 48 times the 
bandwidth. Thus, to protect lower rate paths and speed 
up the restoration process, the present invention pro- 25 
vides capabilities to group STS-1 and STS-3c rate paths 
that have the same end nodes and traverse the same 
route through the restorable network into larger restora- 
tion "bundles." A "bundle" would be treated as a single 
STS-1 2c or STS-48c rate path for the purpose of pre- 30 
computation. During a real-time restoration, the bundle 
would be restored as a unit. 

[0054] In addition, the monitoring techniques de- 
scribed above would now be applied to the constituent 
paths within the bundles. The restoration can thus be 35 
activated based on the failure or degrade of one, some, 
or all of the constituent signals. It is noted that the ANSI 
Tandem Connection capabilities allow for bundling of 
paths across a network. A Tandem Connection fault is 
declared if any one of the constituent paths has a fault. 40 

REAL-TIME RESTORATION ACTIVATION 
SIGNALING 

[0055] As previously indicated, when multiple primary 45 
service paths share the same restoration bandwidth, 
restoration paths cannot be provisioned before a failure 
occurs. The paths must be set up (i.e., the appropriate 
cross-connects must be made at nodes along the path) 
after the failure has been detected, necessitating a rap- so 
id ; robust and reliable signaling architecture. However, 
there are currently no signaling standards to enable the 
distributed activation and set-up of restoration paths in 
SONET-based mesh networks. 

[0056] The signaling method of the present invention 55 
enables sub-second restoration of the network from a 
single failure, given reasonable assumptions about the 
size of the network (large, carrier-grade network), the 



number of cross-connects expected at a typical node 
during restoration, and the rate at which cross-connects 
can be executed at a node. The signaling method of the 
present invention is compatible with non-conforming 
network elements. In addition, the signaling method of 
the present invention uses bandwidth efficiently be- 
cause it does not require additional bandwidth beyond 
that which is required for the restoration paths. Finally, 
the signaling method is reliable. 

Signaling Architecture - Multiple Replaceable Paths 

[0057] As previously indicated, the present invention 
utilizes a multiple replaceable paths approach, wherein 
a signaling message for restoring a failed path is formu- 
lated by the end node(s) and relayed from restoration 
node to restoration node in bandwidth that is assigned 
to the restoration path by the pre-computation. The sig- 
naling message is routed exactly along the pre-comput- 
ed restoration path. For example, if the pre-computation 
indicates that the restoration path for a given primary 
path will use a given SONET line and be the second 
path multiplexed on that line, then the signaling mes- 
sage to restore that path must be routed on that line via 
the second signaling path multiplexed within the line. As 
previously indicated, however, the restoration path is not 
yet set up (the cross -connects have not been made), so 
the restoration path itself cannot be used to transport 
signaling. 

[0058] Thus, a multiple replaceable paths architec- 
ture is utilized for node-to-node signaling, as illustrated 
in FIG. 7. With the disclosed multiple replaceable paths 
architecture, any two adjacent restoration nodes, for ex- 
ample, nodes H and I, create SONET paths, 710 -i, for 
restoration signaling in the restoration (or "spare") band- 
width that lies between them. (In FIG. 7, four paths, la- 
beled 1,2,3 and 4, are shown.) The restoration nodes 
originate and terminate these SONET paths. The paths 
extend between pairs of adjacent restoration nodes and 
pass transparently through any intervening non-con- 
forming SONET network elements because the non- 
conforming network elements do not terminate the 
paths. 

[0059] Subsequently, when an end node of a primary 
path detects an "in-network" path failure as described 
earlier, it formulates a signaling message that uniquely 
identifies the failed path and requests set-up of the res- 
toration path. The restoration signaling message is 
thereafter relayed from one restoration node to another 
in the overhead or payload of the signaling paths that 
occupy the exact same bandwidth that is subsequently 
used by the restoration path. A number of signaling 
paths may be used in tandem to signal a single path 
failure; in this case, the intermediate restoration nodes 
make signaling routing decisions based on information 
from the pre-computation which they have stored in their 
routing tables. (They can do this since the signaling 
message takes the same route through the network as 
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the restoration path.) 

[0060] Once a signaling message requesting a path 
restoration is transmitted to an adjacent restoration 
node using the overhead or payload of a particular sig- 
naling path, the restoration node that transmitted the 
message makes a cross-connect that replaces that sig- 
naling path with a segment of the restoration path whose 
set-up was requested in the transmitted signaling mes- 
sage. At that point, the restoration path is locked in on 
that link, and the signaling path is no longer available 
for transmitting messages. If another restoration path 
set-up message requiring use of the same signaling 
path is subsequently received at the node (e.g., if there 
were multiple simultaneous failures in the network), that 
message cannot be transported. When a signaling mes- 
sage has been relayed through all intermediate nodes 
on the restoration path and reaches the far-end end 
node, and that end node makes its final cross-connect 
and verifies end-to-end connectivity, the failed path is 
restored. 

[0061] For example, in FIG. 7, the pre-computed res- 
toration path for primary path ABCD will pass through 
nodes A, H, I and D. However, until there is a failure of 
ABCD, this restoration path is not set up. Instead, three 
signaling paths occupy ABCD's restoration bandwidth 
on the links between nodes A and H, H and I, and I and 
D. The overheads or pay loads of these paths are used 
to carry messages signaling the failure of path ABCD, if 
it should fail. For example, signaling path 2 temporarily 
occupies ABCD's restoration bandwidth on the link be- 
tween H and I. Hence, signaling path 2 is designated to 
transport messages about a failure of primary path AB- 
CD between restoration nodes H and I. Once these 
messages have been transmitted , the restoration path 
for ABCD replaces signaling path 2 between nodes H 
and I. In other words, if the restoration signaling had 
been initiated starting at node A, once node H has sig- 
naled node I using signaling path 2, node H will then 
connect the segment of the restoration path between A 
and H to the segment of the restoration path between H 
and I. Nodes H and I contain functionality to route and 
relay messages from incoming signaling paths to out- 
going signaling paths. 

[0062] The p re-computation of the present invention 
totally determines which outgoing signaling path a node 
should use to route a message about the failure of a 
particular primary path. This information is stored in the 
routing tables in the control and memory 77 (FIG. 5) at 
each node and updated whenever a new pre-computa- 
tion is completed. 

[0063] Thus, under the multiple replaceable paths ar- 
chitecture of the present invention, a carrier does not 
need to plan and administer a separate signaling net- 
work. The signaling capabilities are embedded in the 
restoration bandwidth, wherever it exists, until the band- 
width is used to restore a failed service path. A restora- 
tion path follows the same route and occupies the same 
bandwidth as its signaling paths, thus providing reliabil- 



ity. If the restoration path is available, the signaling paths 
are also. 

Signaling Messages 

5 

[0064] As previously indicated, signaling messages 
are formulated by path end nodes when an end node 
detects an in-network path failure and are relayed from 
node to node along the restoration route. Signaling mes- 
10 sages are simple, indicating for example, only that a fail- 
ure has occurred, along with the identifier for the path 
that failed, such as SF-Path#N (i.e., Signal Fail on path 
number N). Simple messages can be used because, to 
implement real-time restoration in accordance with the 
15 present invention, each restoration node must be able 
to do two things. First, the node must be able to control 
its own cross-connect map-that is, it must know which 
cross-connect to make to establish a restoration path 
through its fabric. Second : the node must be able to de- 
20 termine where to route a signaling message it receives 
from another node, i.e., which signaling path should be 
used to forward the message to the next node. The data 
each node requires for both these functions is available 
from the pre-computation. The pre-computation com- 
25 pietely determines the restoration path for any given pri- 
mary path. This pre-computed restoration path is always 
the same regardless of the type of failure or where it 
occurs on the primary path. Hence, a request for set-up 
of the restoration path is implicit in a message that indi- 
30 cates which path failed. 

[0065] Which path fails completely identifies the res- 
toration actions any given node must take. Hence, one 
of the outputs of the pre-computation for each restora- 
tion node is a routing table that associates each unique 
35 path identifier with the cross-connect that must be made 
at that specific restoration node to restore service in the 
event that specific path fails. This table would be popu- 
lated whenever a new pre-computation was completed 
and stored in the node itself in control and memory so 
40 that it could be accessed whenever restoration was in- 
itiated. The information stored in the routing table is also 
used to determine where to route a signaling message 
received from another node. The information can be 
used this way because a cross-connect associates a 
45 path segment on one SONET line (e.g., the signaling 
path carrying an incoming path fail message) with a path 
segment on another SONET line (e.g., the signaling 
path carrying an outgoing path fail message), and the 
restoration path uses the same bandwidth as the sign- 
50 aiing paths. 

[0066] The accuracy of a received signaling message 
can be verified in several ways. For example, a CRC 
code may be appended to the message, or the message 
may be repeated at least three times, with the receiver 
55 making the indicated cross-connect only after having re- 
ceived the same message twice, ft is noted that even 
with the CRC, it is still desirable for the transmitter to 
send the message at least twice, to allow for the case 
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where the first transmission of the signaling message 
was corrupted. 

[0067] Due to the limited bandwidth available in the 
SONET path overhead, the restoration signaling mes- 
sages should remain short. With the multiple replacea- 
ble paths signaling of the present invention, it is possible 
to carry a single restoration message and its associated 
overhead (e.g., for framing and possibly CRC) in about 
five bytes, if paths are identified numerically as follows. 
A unique path number could be assigned to each pri- 
mary path in the network using a simple hierarchical 
numbering scheme that (1) assigns a unique number to 
each restoration node in the network, and (2) identifies 
a restoration path using one of its end node numbers (e. 
g., the higher-numbered end node) plus a unique resto- 
ration path number within that end node. 
[0068] It is noted that the F2 byte, or Path User Chan- 
nel of the SONET path overhead, has been allocated in 
the SONET standards for end user communications be- 
tween SONET STS path terminating network elements. 
The signaling paths used by the present invention exist 
only between the restoration nodes/and the restoration 
processing equipment of the present invention is, in ef- 
fect, the end user of these paths. Thus : placing the sig- 
naling messages in the F2 byte of the SON ET path over- 
head can be supported. It is noted that other alternatives 
exist for transmitting the signaling messages, as would 
be apparent to a person of ordinary skill in the art. 
[0069] It is noted that the proposed signaling paths 
would exist only between restoration nodes that provide 
restoration functionality in accordance with the present 
invention and would not be visible outside the network 
of restoration nodes. Thus, any byte or bytes in the over- 
head of these signaling paths could be used to transport 
the signaling messages, provided their use was not es- 
sential for other purposes. As noted earlier, signaling 
path payload could also be used to transport signaling 
messages. 

PRACTICAL EXAMPLE 

[0070] Consider again the network in FIG. 4. Paths 1 
and 2 are primary service paths. Path 1 is a segment of 
the end-to-end SONET Path 1. SONET Path 1 origi- 
nates/terminates outside the restorable network and is 
initially cross-connected through Nodes A, B, C and D 
within the restorable network. Nodes A and D are the 
provisioned restoration end nodes of primary Path 1. 
Path 2 is a segment of the end-to-end SONET Path 2. 
SONET path 2 originates and terminates outside the 
restorable network and is initially cross-connected 
through Nodes E, F and G. Nodes E and G are the pro- 
visioned restoration end nodes of primary Path 2. The 
restoration path for Path 1 extends from Node A through 
Nodes H, I and D. The restoration path for Path 2 ex- 
tends from Node E through Nodes H, I and G. Path 1 
and Path 2 share restoration bandwidth between H and 
I. 



Action of End Nodes 

[0071] As an example, assume that the line carrying 
Path 1 between Nodes B and C is cut. When this hap- 
5 pens, Nodes B and C both detect LOS and transmit AIS- 
P outward along all paths carried on that line. Hence, 
Node B transmits AIS-P from node B toward node A 
along SONET Path 1, and Node C transmits AIS-P 
along SONET Path 1 from node C toward node D. 
10 Nodes A and D are the provisioned restoration end 
nodes for Path 1 . Hence, they continually monitor for 
indications of failure on the path. When an end node de- 
tects AIS-P on a path exiting the restorable network, the 
end node waits for a provisioned period of time to allow 
15 local protection mechanisms, e.g., linear Automatic Pro- 
tection Switching (APS), to take effect, if they are in 
place. If local protection mechanisms fail to correct the 
problem or If there are no local protection mechanisms, 
the problem persists, and the end node initiates resto- 
re ration signaling in accordance with the present inven- 
tion. 

[0072] To initiate restoration signaling, the end node 
(i) creates a signaling message identifying the path that 
failed, e.g. SF-Path#1 , (ii) does a routing table look-up 

25 to determine which outgoing line and signaling path to 
use for transmitting the signaling message to the next 
node on the restoration path, and (iii) writes the mes- 
sage at least three times into the overhead or payload 
of the signaling path. It is noted that if AIS-P is received 

30 at both end nodes on Path 1 , restoration signaling can 
be initiated and proceeds from both end nodes toward 
the middle of the restoration path simultaneously. Allow- 
ing the restoration to proceed in this manner reduces 
total network restoration time by cutting the signaling 

35 time by as much as one-half. 

[0073] After an end node has transmitted the SF- 
Path#1 message at least three times, the end node gen- 
erates a command internally instructing its processor to 
disconnect SONET Path 1 from the primary path and a 

40 subsequent command to cross-connect SONET Path 1 
to the restoration path. As discussed further below, the 
cross-connect to the restoration path is initially one-way, 
providing transmission from the edge of the restorable 
network toward the center of the restorable network, but 

45 not from the restorable network to the outside. When 
executed, this cross-connect causes the restoration 
path to replace the signaling path from Node A to Node 
H. Node A then flags the cross-connect in its database 
located in control and memory 77 (FIG. 5), to indicate 

50 that the associated restoration bandwidth is already in 
use and cannot be used again to restore another failed 
path (e.g., one that shares the same restoration band- 
width) without first taking down the cross-connect that 
has already been made. 

55 [0074] It is important to note that, for rapid restoration 
path set-up, end nodes must initiate signaling to the next 
node along the restoration path before they execute res- 
toration cross-connects. Executing restoration cross- 
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connects disables the signaling paths. In addition, sig- 
naling should proceed as fast as possible across the 
network, allowing some of the cross-connect processing 
at nodes along the restoration route to proceed in par- 
allel. 5 

Action of Intermediate Nodes 

[0075] While the cross-connect command described 
above is being queued and processed at Node A, the 10 
next node on the restoration path, Node H, receives the 
SF-Path#1 message. Receipt of a second, identical 
message that confirms the first message triggers a rout- 
ing table look-up to determine the outgoing line and sig- 
naling path for forwarding the message to the next node 
(Node I) on the restoration path. Restoration signaling 
and cross-connecting can then proceed as at an end 
node, with two notable exceptions. First, there are no 
primary path cross-connections to remove. Second, 
cross-connections made at intermediate nodes are two- 
way cross-connections, i.e. they restore both directions 
of transmission. However, to help prevent misconnec- 
tions of end-user traffic during real-time restoration, the 
cross-connect at an end node must initially be a one- 
way cross-connect so that there is transmission toward 
the center of the restorable network from the edge of the 
network, but not from inside the restorable network to 
the outside. An end node executes a second one-way 
cross-connect allowing transmission in both directions 
only after verifying that the identity of the path it is trans- 
mitting into the restorable network is the same as the 
identity of the path it is receiving from the restorable net- 
work. This may be done by comparing their SONET 
Path Traces (J1-byte), as described below. 
[0076] It is noted that end user SONET Path Termi- 
nating Equipment is not required to load identifying in- 
formation into the Path Trace bytes. The end user may 
insert null characters into the Path Trace bytes instead. 
Therefore, to be able to provide the Path Trace compar- 
ison described above on restorable paths, during real- 
time restoration, restoration end nodes must insert in- 
formation into the J1 -byte of the restoration path to iden- 
tify the primary path being restored. The identifier would 
be the same one used in the signaling messages, which 
is unique to each path. The end node continues to insert 
the primary path identifier into J1 until either (i) it verifies 
that the path it is transmitting into the network has the 
same identifier as the one it is receiving from the net- 
work, or (ii) it times out. In the case of (i), the end node 
generates a command internally to execute the second 
direction of the end-node cross-connect, thus allowing 
end-user traffic to flow in both directions. In the case of 
(ii) the node functions as described below in the section 
entitled "Handling Misconnections." 
[0077] As previously indicated, the present invention 
requires some capabilities that are also required by the 
ANSI Tandem Connection Maintenance standard, 
T1. 105. 05-1 994. It is not necessary, however, to imple- 



ment all the capabilities required for ANSI Tandem Con- 
nection to implement the present invention. Nonethe- 
less, if a full suite of ANSI Tandem Connection capabil- 
ities were available, including the Data Link, some of the 
proposed capabilities for the present invention would 
likely change. In particular the primary path from end 
node to end node would be identified as a Tandem Con- 
nection, and the comparison of Path Traces described 
above would be replaced with comparison of Tandem 
Connection Traces. 

Handling Misconnections 

[0078] If within a pre-determined amount of time : e.g. 
2 seconds after path restoration has been initiated, an 
end node determines that the trace of the received path 
is still not equal to the trace of the transmitted path, then 
the assumption is restoration has not occurred. There 
may in fact, be a misconnection of restoration paths. 
Most misconnections will likely occur as a result of mul- 
tiple, simultaneous failures of paths sharing the same 
restoration bandwidth. Misconnections will not be visible 
to end users because end nodes on the restoration path 
verify that the received and transmitted Path Traces are 
identical before completing their restoration cross-con- 
nections and allowing traffic to flow. However, when mis- 
connections do occur, they leave the network in an un- 
known state, which is undesirable for network providers. 
Therefore, misconnections must be removed and the 
30 network restored to a known state. 

[0079] When there is a misconnection, three or more 
end nodes may be involved. To remove misconnections, 
the affected end nodes transmit signaling messages to 
take down whichever parts/segments of the (miscon- 
35 nected) restoration paths have been set up. Since mis- 
connections during real-time restoration are expected 
to be infrequent, recovery from them is not considered 
as time critical as setting up restoration paths. Hence, 
signaling to take down misconnected path segments 
40 can use OSI messaging over the SONET Section Data 
Communications Channel (DCC). Use of the SONET 
Section DCC is desirable because the SONET signaling 
paths used initially to set up (misconnected) restoration 
paths are no longer in place along the entire lengths of 
45 the restoration routes. They have been replaced by seg- 
ments of actual restoration paths transmitting end cus- 
tomer path payload and overhead. Hence, the end cus- 
tomer has unrestricted access to these restoration paths 
and overhead, and as a result, their use for restoration 
so signaling is undesirable from a security perspective. 
[0080] To take down misconnected path segments 
using OSI messaging over the SONET Section DCC, 
each affected end node (i) identifies the restoration path 
to be removed (e.g., ResPath#5), (ii) determines the 
55 next node on this path, and (iii) sends a remove path 
message, e.g., RM-ResPath#5, to that node using OSI 
messaging and routing on the SONET DCC. It is noted 
that the next node on this path is derived from two sourc- 
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es. The first is a table associating all links on the node 
with the identity of the adjacent restoration node to 
which those links are connected. This table should be 
available in control and memory since its information is 
required input for the pre-computation. The second is $ 
the routing table 77described earlier, that is the output 
of the pre-computation. 

[0081] After sending the remove message, the end 
node does a restoration table look-up to determine 
which local cross-connect must be removed (i.e., dis- io 
connected) to take down ResPath#5. The end node 
then generates and executes a command that removes 
the identified cross-connect. If that cross-connect is not 
in place (e.g., it was taken down by a previous command 
from another source), the remove command is denied, is 
In either case, ResPath#5 no longer flows through the 
node. After the restoration path cross-connect is re- 
moved, the end node re-establishes the signaling path 
in the restoration bandwidth. 

[0082] Each intermediate node that subsequently re- 20 
ceives the remove path message determines the next 
node on the restoration path and forwards the remove 
message to that node via the DCC. After forwarding the 
message, the node queries its own restoration table to 
determine which local cross-connect to remove, gener- 25 
ates and executes a command that removes it, and 
sends a confirmation to the node from which it received 
the remove message. (If the node finds the cross-con- 
nect that is to be removed is not in place, it simply denies 
the remove command.) The node then re-establishes 30 
the signaling path in the restoration bandwidth previous- 
ly occupied by the (misconnected) restoration path. 
When the far-end node finally receives the remove path 
message, it queries its tables and takes down the local 
cross-connect. The misconnection is effectively re- 35 
moved when the remove messages have been proc- 
essed at all nodes on all affected restoration paths and 
the corresponding signaling paths have been reestab- 
lished. 

[0083] After clearing a misconnection, end nodes can 40 
re-initiate signaling to restore affected failed paths. To 
help prevent the same misconnection from occurring a 
second time, each of the end nodes waits for a randomly 
selected length of time before attempting to set-up a res- 
toration path again. 45 

Handling Paths With Priorities 

[0084] In the processor of a restoration node : resto- 
ration cross-connect commands would normally be so 
queued and processed in the order in which they are 
generated. Once a restoration path has been set up for 
one failed path, it will not be possible to "bump" that path 
off the restoration bandwidth and replace it with another 
path having higher priority (e.g., as in an analogous 1 XN ss 
linear APS). However, it is possible to provide high-pri- 
ority paths with some degree of priority in the restoration 
processing. This could be done by creating two queues 



at each node, one for high-priority cross-connects (i.e., 
those associated with failed high-priority paths) and one 
for standard cross-connects (i.e., those not associated 
with high-priority paths). As long as there are cross-con- 
nect commands in the high-priority queue, the proces- 
sor acts on these first. When the high-priority queue is 
empty, the processor processes standard cross-con- 
nect commands. Although this procedure does not guar- 
antee that high-priority paths will be restored before 
standard paths, it raises the probability that at any given 
node a high priority path will seize restoration bandwidth 
before a lower priority path, thus increasing the overall 
probability that high priority paths will be restored first. 
[0085] It is also noted that if a primary path absolutely 
must be restored in the event of a failure, then It can be 
assigned a dedicated restoration path during the pre- 
computation. Since the restoration path is dedicated, It 
may be provisioned at the same time the primary path 
is provisioned and service permanently bridged onto it. 
With a permanent bridge at one end of the path and a 
selector function at the other, ring-like restoration 
speeds (i.e., 50 msec) can be achieved. 

Disconnecting Restorable Paths 

[0086] It will sometimes be necessary to remove a 
restorable primary path that has no faults or perform- 
ance degradation, e.g., when a customer's service is 
disconnected. However, attempting to take down a 
restorable path by removing individual cross-connects 
along the primary path would likely result in the end 
nodes initiating signaling to set up the restoration path. 
Hence, restoration nodes should deny technician-gen- 
erated commands to remove individual cross-connects 
at a node if the cross-connects are part of a restorable 
path. 

[0087] Instead, the signaling technique discussed 
earlier for removing misconnections should be used to 
remove restorable paths. In this case, however, the sig- 
naling is triggered by a technician's command to remove 
a path, e.g., RM-Path#7, entered atone of the path end 
nodes. The end node puts a flag on the path to inhibit 
restoration signaling from being triggered, and then be- 
gins the process of removing Path#7 by doing a table 
look-up to determine the next node on the path. Signal- 
ing and removal of cross-connects can proceed In the 
same way as described for removing misconnections. 

Capabilities for Multiple Replaceable Paths on 
Restoration Nodes 

[0088] To provide the functions described above for 
real-time node-to-node signaling using the multiple re- 
placeable paths signaling architecture, the capabilities 
of the restoration nodes must include the ability to: 

(i) monitor for path failures at the end nodes, as de- 
scribed earlier, 
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(ii) originate and terminate SONET paths for use in 
signaling in the restoration bandwidth between two 
restoration nodes, including establishing/locating 
SONET Path Pointers and Overhead bytes. These 
SONET paths should be active in the restoration (or 5 
spare) bandwidth' when there is no restoration path 
occupying the bandwidth, 

(iii) provide read and write access to the Path User 
Channel (F2) or some other acceptable path over- 
head byte or bytes, or to path payload, for the pur- 10 
pose of transmitting real-time restoration signaling 
messages on the signaling paths described above, 

(iv) incorporate and then populate signaling routing 
tables and other restoration tables, using data that 

is either downloaded from a centrally calculated *s 
pre-computation or from a distributed pre-computa- 
tion ; 

(v) route real-time restoration path set-up messag- 
es based on received message content and subse- 
quent routing table look-up, 20 

(vi) format messages to remove paths and to con- 
firm their removal, and to route these messages us- 
ing OSI on the SONET Section DCC, 

(vii) internally generate either a cross-connect or 
disconnect command based on information ob- 25 
tained from the content of a received signaling mes- 
sage and subsequent restoration table look-up, 
queue and execute the command, 

(viii) if an end node, transmit the identity of the pri- 
mary path being restored into the path trace 30 
(J1 -byte) of the restoration path, 

(ix) if an end node, monitor the restoration path 
trace in the received direction and compare it with 
the transmitted path trace for instances of miscon- 
nections and other anomalies, as well as for sue- 35 
cessful restoration of failed or degraded paths, 

(x) if an end node, initiate back-out procedures in 
the event of a misconnect, 

(xi) if an end node, initiate removal of paths on com- 
mand from a user interface, and 40 

(xii) provide priority processing, as described earlier 
in this document. 

[0089] It is to be understood that the embodiments 
and variations shown and described herein are merely 
illustrative of the principles of this invention and that var- 
ious modifications may be implemented by those skilled 
in the art without departing from the scope and spirit of 
the invention. 



Claims 

1 . A method for restoring a service path in a network, 
said service path having a pre-computed restora- 55 
tion path, said pre-computed restoration path hav- 
ing at least one segment, said method comprising 
the steps of: 
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detecting a restorable failure along said service 
path; and 

signaling the restoration of said failure using at 
least one signaling path that occupies the same 
bandwidth as said pre-computed restoration 
path, each of said at least one signaling paths 
being replaced by a segment of said pre-com- 
puted restoration path after signaling is com- 
plete. 

2. A method for restoring a service path in a network, 
said service path having a pre-computed restora- 
tion path, said pre-computed restoration path hav- 
ing at least one segment, said method comprising 
the steps of: 

detecting a failure along said service path; 
determining if said failure is a restorable failure; 
signaling the restoration of said restorable fail- 
ure using at least one signaling path that fol- 
lows said pre-computed restoration path, said 
pre-computed restoration path segments re- 
placing said at least one signaling paths after 
signaling is complete; and 
connecting said pre-computed restoration 
path. 

3. The method of claim 1 or 2 wherein said network is 
a SONET network. 

4. The method of claim 1 or 2 wherein said network is 
an SDH network. 

5. The method of claim 1 or 2 wherein said network is 
an optical network having nodes capable of access- 
ing digital overhead on said paths. 

6. The method of claim 1 or 2 wherein said signaling 
step is initiated by an end-node along said service 
path. 

7. The method of claim 1 or 2 further comprising the 
step of distinguishing a restorable failure from a 
non-restorable failure to determine whether to acti- 
vate said restoration. 

8. The method of claim 1 or 2 wherein a signaling mes- 
sage is transmitted in an overhead portion of said 
at least one signaling path. 

9. The method of claim 1 or 2 wherein a signaling mes- 
sage is transmitted in a payload portion of said at 
least one signaling path. 

10. The method of claim 1 or 2 wherein a signaling mes- 
sage identifies said service path having said failure 
and requests the establishment of said pre-comput- 
ed restoration path. 
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1 1 . The method of claim 1 0 wherein said signaling mes- 
sage is relayed from one restoration node to anoth- 
er node in the overhead or payload of said at least 
one signaling path, based on an identity of a failed 
service path identified in said signaling message. 5 

12. The method of claim 11 further comprising the step 
of establishing a cross-connect that replaces said 
at least one signaling path with a segment of the 
pre-computed restoration path requested in the sig- 10 
naling message, said establishing step being per- 
formed after relaying said signaling message to a 
subsequent restoration node. 

13. The method of claim 1 or 2 wherein said signaling is 
path transits non-conforming network elements. 

1 4. The method of claim 1 or 2 wherein a non-restorable 
failure is indicated using a flag in a path overhead 
field. 20 

15. The method of claim 1 or 2 further comprising the 
step of determining if said failure is a restorable fail- 
ure using criteria from the ANSI Tandem Connec- 
tion Maintenance standard. 25 

16. The method of claim 1 or 2 wherein said network is 
a restorable network within a larger multi-network 
environment and wherein said signaling step is ini- 
tiated only when the fault causing said path failure 30 
is located within the restorable network. 

17. The method of claim 1 or 2 wherein customer path 
terminating equipment is not part of a restorable 
network, and wherein said signaling step is initiated 35 
only when the fault causing said path failure is lo- 
cated within said restorable network. 

18. The method of claim 1 or 2 wherein adjacent resto- 
ration nodes in said network initiate and terminate *o 
paths that are used for signaling in spare network 
bandwidth, wherein said signaling paths remain in 
place for signaling until replaced by said pre-com- 
puted restoration paths used to restore service. 

45 

19. The method of claim 1 or 2 wherein end nodes are 
identified for said service path when said service 
path is initially provisioned. 

20. The method of claim 19 wherein said end nodes so 
monitor for said path failures and initiate restoration 
signaling only when said path failure is due to a fault 
located between the end nodes. 

21. The method of claim 19 wherein said end nodes (i) 55 
formulate a restoration message uniquely identify- 
ing said failed service path and requesting set-up 

of said pre-computed restoration path, and (ii) route 



said message to a subsequent restoration node. 

22. The method of claim 19 wherein said end nodes 
permits traffic to flow out of the network on a re- 
stored path only after verifying both end node-to- 
end node connectivity and an identity of the re- 
stored path. 

23. A system for restoring a service path in a network, 
said service path having a pre-computed restora- 
tion path, said pre-computed restoration path hav- 
ing at least one segment, said system comprising: 

a memory for storing computer-readable code; 
and 

a processor operatively coupled to said mem- 
ory, said processor configured to: 

detect a restorable failure along said serv- 
ice path; and 

signal the restoration of said failure using 
at least one signaling path that occupies 
the same bandwidth as said pre-computed 
restoration path, each of said at least one 
signaling paths being replaced by a seg- 
ment of said pre-computed restoration 
path after signaling is complete. 

24. A system for restoring a service path in a network, 
said service path having a pre-computed restora- 
tion path, said pre-computed restoration path hav- 
ing at least one segment, said system comprising: 

a memory for storing computer-readable code; 
and 

a processor operatively coupled to said mem- 
ory, said processor configured to: 

detect a failure along said service path; 
determine if said failure is a restorable fail- 
ure; 

signal the restoration of said restorable fail- 
ure using at least one signaling path that 
follows said pre-computed restoration 
path, said pre-computed restoration path 
segments replacing said at least one sign- 
aling paths after signaling is complete; and 
connect said pre-computed restoration 
path. 
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